Why is miller rabin still public? I would prefer something else here.

• Only make mp_is_prime public
• Private s_mp_prime_fermat, s_mp_prime_miller_rabin, s_mp_prime_frobenius_underwoord, s_mp_prime_lucas_selfridge, s_mp_prime_is_divisible (maybe rename to s_mp_prime_trial_divisions).
• mp_is_prime should call the s_mp_prime functions depending on which are enabled by checking MP_HAS
• The first s_mp_prime function to check should be the best one (lucas selfridge?)
• The user can then produce a smaller version of the library by disabling the larger tests or only enabling the specific test that is desired.
• There is a test in mp_prime_is_prime which checks if the number is equal to one in the prime table. This test should be moved to s_mp_prime_is_divisible.
• Get rid of LTM_USE_ONLY_MR, LTM_USE_FROBENIUS_TEST

Ping @czurnieden @sjaeckel

Why is miller rabin still public?

Because it is useful as a stand-alone function:

• you can make a deterministic test (the original MIller-test) without much ado. Upper limit of witnesses is rather low assuming ERH.
• you can build a very small and cryptographically secure, albeit slow, probabilistic test.

< make [...] mp_prime_is_divisible private

In it's reduced function (only a small table, instead of a sieve): yes.

mp_is_prime should call the s_mp_prime functions depending on which are enabled by checking MP_HAS

Tempting but mp_is_prime is a package with a well defined behaviour; you cannot arbitrarily switch (on/off/exchange) components.

The first s_mp_prime function to check should be the best one (lucas selfridge?)

They work in combination to make the BPSW test. The order does not matter much (although you need to put the is_square test before M-R and L-S tests) but using runtime order, fastest first, seems to make the most sense to me.

There is a test in mp_prime_is_prime which checks if the number is equal to one in the prime table. This test should be moved to s_mp_prime_is_divisible.

Yeah, the whole thing needs a bit of a clean-up, admitted, but cryptographic functions are quite sensible and should not be touched without a very good reason. What is your reason to move that test?

Get rid of LTM_USE_ONLY_MR, LTM_USE_FROBENIUS_TEST

The Frobenius-Underwood test is gone for good (was only needed for MP_8BIT).
On the other side: the Frobenius-Underwood test is independant of the BPSW test, we could do a "BPSW 'extra paranoid'" version but I don't want to maintain so many functions, especially if they are of cryptographic nature needing a large amount of extra scrutiny.

The LTM_USE_ONLY_MR branching is not very elegant and can be misleading. I was thinking about removing it completely and replace it with two or three complete functions offering Miller-Rabin only and BPSW. Most of the two functions overlap and can be put in external (private) functions such that we get

Miller-Rabin full:

1. preamble (trial division, is-square, etc)
2. Miller-Rabin test with a number of random witnesses (probabilistic) or a sequence of witnesses (probabilistic/deterministic). Those two can be split into two individual (private) functions, too.

BPSW:

1. preamble (trial division, is-square, etc)
2. Miller-Rabin test with witnesses 2 and 3
3. strong Lucas-Selfridge test

BPSW short:
2. Miller-Rabin full test with witnesses 2 and 3
3. strong Lucas-Selfridge test

BPSW "paranoid" (as it is now):

1. BPSW short
2. Miller-Rabin with a number n > 0 of random witnesses

Where BPSW "paranoid" would be the algorithm for mp_is_prime and mp_is_prime will be the only function in LTM to be called "cryptograhically secure" for reasons of CYOA.

If we make a function mp_prime_preamble all of the functions above are very small, just calling the preamble and one or two tests.

You can fiddle with the preamble as long as the result is an odd number larger than two. The condition of non-square is not necessary for MIller-Rabin but would need an additional round which is on average more costly than the is-square test and the Lucas-Selfridge test doesn't like squares either but it is not technically needed.

PS: there seems to be an interesting ansatz to proof the independence of MIller-Rabin and Lucas tests by Alec Leng but it dates March 2017 and I found no follow-up. Asking himself? Yeah, might do.

Because it is useful as a stand-alone function: you can make a deterministic test (the original MIller-test) without much ado. Upper limit of witnesses is rather low assuming ERH. you can build a very small and cryptographically secure, albeit slow, probabilistic test.

Why do I need it when I have the mp_is_prime function which does things better? By allowing configuration of the components, expert users can switch out the test. But I think it would be best to provide an API which fits to the common usage pattern. I don't understand why Miller-Rabin is so special that it should be exposed as a single test in contrast to the other ones.

Tempting but mp_is_prime is a package with a well defined behaviour; you cannot arbitrarily switch (on/off/exchange) components.

It has been changed before I think. This time we would not change the defaults, but would allow internal selection of other tests if that is desired. The default as of now is BPSW or the even the paranoid variant? By adjusting the configuration the function can be modified (as can many other parts of the library). The user has to know what they are doing in that case. But the configuration mechanism is a very internal thing since 2.0 at least, so I am not worried about that. And we should probably put another warning to the documentation regarding configuration.

Yeah, the whole thing needs a bit of a clean-up, admitted, but cryptographic functions are quite sensible and should not be touched without a very good reason. What is your reason to move that test?

Yes one has to be careful but I think it would be nice to have this prime table used by only one function.
s_mp_prime_is_divisible should return three values then (IS_PRIME if in the table, IS_NOT_PRIME if divisible by one of the primes in the table, UNDECIDED in the other cases).

Obsoleted by #541
Please move any further discussion there.